Skip to content

Mig release 5.7.0 rc8#10

Draft
mhaque07 wants to merge 371 commits intoKB-iGOT:mig-cbrelease-4.8.16_RC6from
mhaque07:mig-release-5.7.0_RC8
Draft

Mig release 5.7.0 rc8#10
mhaque07 wants to merge 371 commits intoKB-iGOT:mig-cbrelease-4.8.16_RC6from
mhaque07:mig-release-5.7.0_RC8

Conversation

@mhaque07
Copy link
Copy Markdown

@mhaque07 mhaque07 commented Jun 2, 2025

No description provided.

santhosh-tg and others added 30 commits November 9, 2022 20:05
…thosh-tg/release-5.0.0

Release 5.0.0 - GCP related changes
…akartheekreddy/csp-migration

Issue #KN-603 fix: CSP changes
…ect-sunbird/csp-migration

Issue #KN-603 merge: Merge CSP migration change to release-5.1.0
Signed-off-by: Keshav Prasad <keshavprasadms@gmail.com>
…shavprasadms/release-5.1.0

fix: SB-31155  using cloud_storage_url instead of hard coded url
# Conflicts:
#	kubernetes/ansible/roles/flink-jobs-deploy/defaults/main.yml
…akhil46/release-5.1.0

Added ES related roles
pallakartheekreddy and others added 30 commits June 21, 2023 18:25
…e-5.6.0

Issue KN-10 fix: disable content publish, unlisted publish, reject and review APIs
…prakash8887/release-5.4.1

Issue #KN-889 fix: QR ImageURL bulk sync
Issue #IQ-493 feat: added code for quml migration event generation
* Add task to start neo4j

* merge Release 5.5.0 into 5.6.0 (Sunbird-Knowlg#1971)

* Issue #KN-427 feat: Knowledge service cloud-agnostic

* added artifact upload  for oci oss storage

Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com>

* owerwrite oss object if already exists

Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com>

* testing with artifact download for neo4j from oci oss

Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com>

* changed the oss object download command

Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com>

* added checkpoint

Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com>

* removed checkpoint for debuging

Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com>

* added a cloud storage type selector for oci

Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com>

* jinja indentation

Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com>

* jinja indentation correction

Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com>

* removed the extra line for cloud_storage_type

Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com>

* added s3 credential configuration for flink

Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com>

* disabled s3 storage temporarily

Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com>

* s3 config for flink jobmanager only

Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com>

* corrected if else syntax

Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com>

* added s3 config for taskmanager

Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com>

* added base url configuration for s3

Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com>

* jinja templating syntax error correction

Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com>

* corrected base url for s3

Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com>

* updated flink-conf for questionset-publish

Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com>

* testing with s3 configuration

Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com>

* endpoint url changed home region

Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com>

* updated values.j2
cloud_service_provider
cloud_storage_endpoint

Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com>

* added closing (") for variable cloud_service_provider

Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com>

* added hadoop-fs parameters to flink-conf for questionset

Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com>

* corrected the flink-conf for questionset

Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com>

* updated presto specific variables in flink-conf.yaml

Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com>

* added s3-fs-hadoop values in flink-conf for question set

Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com>

* updated the correct values for flink-conf

Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com>

* removed values form flink-conf.yaml

Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com>

* updated the deployment template

Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com>

* added hive.s3.path-style-access

Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com>

* removed hive access path

Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com>

* template value for path.style

Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com>

* added s3.aws.cred provider

Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com>

* removed s3.aws.credentials.provider

Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com>

* testing with flink-conf.yaml

Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com>

* added s3.aws.credentials.provider

Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com>

* updated value of s3.aws.credentials.provider

Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com>

* removed s3.aws.credentials.provider

Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com>

* added s3.aws.credentials.provider for asset-enrichment

Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com>

* added acces key for asset enrichment

Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com>

* added provider in the jobmanager deployment

Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com>

* removed aws provider from deployment and asset-enrichment

Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com>

* added aws cred provider in job mgr deployment

Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com>

* testing with signing algo

Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com>

* testing with S3SignerType

Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com>

* update AWS3SignerType

Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com>

* removed signing algorithm

Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com>

* trying with s3a

Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com>

* reverted to s3

Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com>

* AWS4SignerType algo

Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com>

* AWS4UnsignedPayloadSignerType

Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com>

* NoOpSignerType algo

Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com>

* S3SignerType

Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com>

* removed signer algo

Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com>

* added aws logging

Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com>

* added http logging

Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com>

* added flink debug logs

Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com>

* added http logging

Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com>

* removed http logging

Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com>

* testing bucket probing

Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com>

* bucketprobe=2

Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com>

* bucketprobe=1

Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com>

* disable bucket probe

Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com>

* removed bucket probe

Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com>

* testing with disabing bucket probe

Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com>

* passing the region to hadoop configuration

Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com>

* presto config

Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com>

* added the region to deployment

Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com>

* updated taskmanager with presto variables

Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com>

* removed oss connection details

removed from flink-conf.yaml for questionset-publish as it no longer required

Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com>

* Added oci cloud-store-sdk
updated artifact id to cloud-store-sdk_2.11
version: 1.45

Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com>

* cloud-store-sdk with exclusion

Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com>

* removed exlusion from cloud-store-sdk dependency

Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com>

* rearranged modules for service

Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com>

* reverted modules ordering

Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com>

* added exclusion for guava for cloud-store-sdk

Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com>

* removed exlusion for guava

Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com>

* updated cassandra driver version to 3.2.0
This is test guava 20

Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com>

* update cloud-store-sdk version to 1.4.6

Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com>

* passing an empty region

in cloud-store-sdk 1.4.6 region is defined as an Option<String> , so need to pass it as a optional string

Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com>

* updated cassandra driver to 3.1.2

cloud-store-sdk 1.4.6 uses guava 19, so reverted the change from 3.2.0

Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com>

* Updated the oss_bucket_name

* Updated the instance principal

* Added  env var for OCI

* Corrected the oss bucket name key

* added oci specs for video-stream deployment for kp flinkjobs

Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com>

* updated the zookeeper ip from local host to the ingestion and processing

Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com>

* disabled service_fact gathering

Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com>

* disabled service_facts

Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com>

* Update values.j2

To add the media workflow id

* Issue #KN-0000 feat: Config update

* Issue #KN-0000 feat: Config update

* Issue #KN-0000 feat: Config update

* Issue #KN-0000 feat: Config update

* added endpoint to flinkjob configuration

Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com>

* added the alter replication and partition

Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com>

* testing list length condition

Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com>

* added condition to create topic

Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com>

* removed replication factor alter

Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com>

* oci media streaming changes

* added cloud_storage_type key

Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com>

* udpated cloud-store-sdk id

Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com>

* resolved conflicts

Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com>

* added cloud_storage_proxy_host for qrcode job

Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com>

* added value for proxy host

Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com>

* Issue #KN-195 feat: CSP migrator parallelism increase

* To troubleshoot download content issue

* Adding the missing property

* Reverting the changes made in this deploy branch

* missing property

* for testing publish unit test

Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com>

* updated the continer name value for unit test case

Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com>

* testing unit tests

Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com>

* updated endpoint for unit testing

Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com>

* youtube chcks

Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com>

* for testing unit case with azure

Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com>

* testing unit cse

Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com>

* added cloud ep for unit test

Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com>

* adding the missing config

* updated generic variables

Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com>

* for test runs in circleci

Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com>

* updated test data

Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com>

* updated youtube link

Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com>

* updated invalid youtube links

Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com>

* updated youtube link for test cases

Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com>

* testing creative commons lic

Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com>

* passing mime type field for test case

Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com>

* reverted the mimetype value

Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com>

* KN-CSP Changes

* KN-CSP Changes

* KN-Fix For CSP Changes

* KN-Fix For CSP Changes

* KN-Fix For CSP Changes

* KN-Fix For CSP Changes

* KN-Fix For CSP Changes

* KN-Fix For CSP Changes

* KN-Fix For CSP Changes

* KN-Fix For CSP Changes

* KN-Fix For CSP Changes

* KN-Fix For CSP Changes

* Issue #KN-920 fix: Update kubernetes/ansible/roles/flink-jobs-deploy/defaults/main.yml

Co-authored-by: Mahesh Kumar Gangula <mahesh@sanketika.in>

* Revert "KN-CSP Changes"

* Issue KN-921 fix: config issue fix

* Issue KN-921 fix: config issue fix

* Issue KN-921 fix: config issue fix

* Issue KN-921 fix: AWS support for Flink jobs

---------

Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com>
Co-authored-by: Jayaprakash8887 <Jayaprakash.narayanaswamy@tarento.com>
Co-authored-by: vinukumar-vs <vinukumar_vs@yahoo.com>
Co-authored-by: Deepak Devadathan <deepak.devadathan@oracle.com>
Co-authored-by: Kenneth Heung <kenneth.heung@gmail.com>
Co-authored-by: subhash_chandra_budde <subhash.chandra.budde@oracle.com>
Co-authored-by: rjanart <123344037+rjanart@users.noreply.github.com>
Co-authored-by: Anil Gupta <ani2824@gmail.com>
Co-authored-by: Ramya <ramya.janarthanan@oracle.com>
Co-authored-by: Aiman Sharief <aiman.sharief.17@gmail.com>
Co-authored-by: Mahesh Kumar Gangula <mahesh@sanketika.in>

---------

Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com>
Co-authored-by: santhosh-tg <santhoshkumar.sreedharan@tarento.com>
Co-authored-by: Jayaprakash8887 <Jayaprakash.narayanaswamy@tarento.com>
Co-authored-by: vinukumar-vs <vinukumar_vs@yahoo.com>
Co-authored-by: Deepak Devadathan <deepak.devadathan@oracle.com>
Co-authored-by: Kenneth Heung <kenneth.heung@gmail.com>
Co-authored-by: subhash_chandra_budde <subhash.chandra.budde@oracle.com>
Co-authored-by: rjanart <123344037+rjanart@users.noreply.github.com>
Co-authored-by: Anil Gupta <ani2824@gmail.com>
Co-authored-by: Ramya <ramya.janarthanan@oracle.com>
Co-authored-by: Aiman Sharief <aiman.sharief.17@gmail.com>
Co-authored-by: Mahesh Kumar Gangula <mahesh@sanketika.in>
Issue #IQ-670 fix: fix for selective quml migration
Issue #KN-1089 fix: added default value for endpoint variable
Issue #KN-1089 fix: added default value for proxy host
…n-event-processor

transaction-event-processor config added
…n-event-processor

transaction-event-processor config added
…mediakind

Issue KN-1096 feat: Video streaming job MediaKind configurations added
Issue KN-1096 feat: Removed unused AMS variables
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.